26 research outputs found
A Convex Feature Learning Formulation for Latent Task Structure Discovery
This paper considers the multi-task learning problem and in the setting where
some relevant features could be shared across few related tasks. Most of the
existing methods assume the extent to which the given tasks are related or
share a common feature space to be known apriori. In real-world applications
however, it is desirable to automatically discover the groups of related tasks
that share a feature space. In this paper we aim at searching the exponentially
large space of all possible groups of tasks that may share a feature space. The
main contribution is a convex formulation that employs a graph-based
regularizer and simultaneously discovers few groups of related tasks, having
close-by task parameters, as well as the feature space shared within each
group. The regularizer encodes an important structure among the groups of tasks
leading to an efficient algorithm for solving it: if there is no feature space
under which a group of tasks has close-by task parameters, then there does not
exist such a feature space for any of its supersets. An efficient active set
algorithm that exploits this simplification and performs a clever search in the
exponentially large space is presented. The algorithm is guaranteed to solve
the proposed formulation (within some precision) in a time polynomial in the
number of groups of related tasks discovered. Empirical results on benchmark
datasets show that the proposed formulation achieves good generalization and
outperforms state-of-the-art multi-task learning algorithms in some cases.Comment: ICML201
A dual framework for low-rank tensor completion
One of the popular approaches for low-rank tensor completion is to use the
latent trace norm regularization. However, most existing works in this
direction learn a sparse combination of tensors. In this work, we fill this gap
by proposing a variant of the latent trace norm that helps in learning a
non-sparse combination of tensors. We develop a dual framework for solving the
low-rank tensor completion problem. We first show a novel characterization of
the dual solution space with an interesting factorization of the optimal
solution. Overall, the optimal solution is shown to lie on a Cartesian product
of Riemannian manifolds. Furthermore, we exploit the versatile Riemannian
optimization framework for proposing computationally efficient trust region
algorithm. The experiments illustrate the efficacy of the proposed algorithm on
several real-world datasets across applications.Comment: Aceepted to appear in Advances of Nueral Information Processing
Systems (NIPS), 2018. A shorter version appeared in the NIPS workshop on
Synergies in Geometric Data Analysis 201
Efficient Output Kernel Learning for Multiple Tasks
The paradigm of multi-task learning is that one can achieve better
generalization by learning tasks jointly and thus exploiting the similarity
between the tasks rather than learning them independently of each other. While
previously the relationship between tasks had to be user-defined in the form of
an output kernel, recent approaches jointly learn the tasks and the output
kernel. As the output kernel is a positive semidefinite matrix, the resulting
optimization problems are not scalable in the number of tasks as an
eigendecomposition is required in each step. \mbox{Using} the theory of
positive semidefinite kernels we show in this paper that for a certain class of
regularizers on the output kernel, the constraint of being positive
semidefinite can be dropped as it is automatically satisfied for the relaxed
problem. This leads to an unconstrained dual problem which can be solved
efficiently. Experiments on several multi-task and multi-class data sets
illustrate the efficacy of our approach in terms of computational efficiency as
well as generalization performance